Evaluation of multi-assay algorithms for identifying individuals with recent HIV infection: HPTN 071 (PopART)

Background Assays and multi-assay algorithms (MAAs) have been developed for population-level cross-sectional HIV incidence estimation. These algorithms use a combination of serologic and/or non-serologic biomarkers to assess the duration of infection. We evaluated the performance of four MAAs for individual-level recency assessments. Methods Samples were obtained from 220 seroconverters (infected <1 year) and 4,396 non-seroconverters (infected >1 year) enrolled in an HIV prevention trial (HPTN 071 [PopART]); 28.6% of the seroconverters and 73.4% of the non-seroconverters had HIV viral loads ≤400 copies/mL. Samples were tested with two laboratory-based assays (LAg-Avidity, JHU BioRad-Avidity) and a point-of-care assay (rapid LAg). The four MAAs included different combinations of these assays and HIV viral load. Seroconverters on antiretroviral treatment (ART) were identified using a qualitative multi-drug assay. Results The MAAs identified between 54 and 100 (25% to 46%) of the seroconverters as recently-infected. The false recent rate of the MAAs for infections >2 years duration ranged from 0.2%-1.3%. The MAAs classified different overlapping groups of individuals as recent vs. non-recent. Only 32 (15%) of the 220 seroconverters were classified as recent by all four MAAs. Viral suppression impacted the performance of the two LAg-based assays. LAg-Avidity assay values were also lower for seroconverters who were virally suppressed on ART compared to those with natural viral suppression. Conclusions The four MAAs evaluated varied in sensitivity and specificity for identifying persons infected <1 year as recently infected and classified different groups of seroconverters as recently infected. Sensitivity was low for all four MAAs. These performance issues should be considered if these methods are used for individual-level recency assessments.


Introduction
Characterization of individuals with recent HIV infection can provide important information about the state of the HIV/AIDS epidemic, inform the impact of interventions for HIV prevention, and help characterize populations at increased risk of infection [1]. Recently-infected persons may also be more likely to transmit HIV to others, especially if they are not aware of their infection status and are not yet on antiretroviral treatment (ART) [2]. Multi-assay algorithms (MAAs, also referred to as Recent Infection Test Algorithms [RITAs]) have been developed for estimating population-level incidence in cross-sectional surveys; these algorithms use two or more assays to classify samples as "recent" vs. "non-recent" [3][4][5][6][7][8][9]. MAAs have also been used in research studies to identify and characterize individuals with recent HIV infection [10][11][12][13]. Other studies have noted the potential to use MAAs in real-time to inform persons of their recency status [14,15], to identify patients with recent infection for clinical purposes [16], or to inform epidemiologic surveillance studies and public health responses to the HIV epidemic [17,18]. However, there are limited data assessing the performance of MAAs for individual-level recency assessments [19].
Most multi-assay algorithms (MAAs) used for cross-sectional HIV incidence estimation include a combination of serologic and non-serologic assays [5][6][7][8][9]. The limiting antigen avidity enzyme immunoassay (LAg-Avidity assay) is the most widely used serologic assay for HIV incidence estimation [20,21]. This assay was developed by the United States (US) Centers for Disease Control (CDC) and is commercially available. A point-of-care, qualitative LAg-Avidity assay is also available. These assays are intended for research use only and are not cleared by the US Food and Drug Administration (FDA) for clinical use [22,23]. The most widely used MAA for cross-sectional incidence estimation includes the LAg-Avidity assay plus HIV viral load [20,24]. In this MAA, low HIV viral load is assumed to be due to ART; persons with low viral loads are classified as not recently infected [20,24]. Individuals who start ART early in infection and achieve viral suppression may be misclassified using this approach [25]. This type of misclassification is likely to increase over time, since ART is now recommended for all HIV-infected persons, regardless of CD4 cell count [26].
Viral suppression can also impact the accuracy of individual serologic incidence assays by altering the antibody response to HIV infection [27][28][29][30]. Individuals who are virally suppressed on ART have low levels of circulating virus. In these persons, the level or avidity of anti-HIV antibodies may be reduced [27], which can impact the performance of serologic incidence assays [28][29][30]. Down-regulation of anti-HIV antibodies has also been observed in HIV controllers, who are virally suppressed in the absence of ART [27]. We recently demonstrated that anti-HIV antibody responses are different in persons with natural vs. ARTinduced viral suppression [31]. It is not known if serologic incidence assays perform differently in individuals with natural vs. ART-induced viral suppression.

PLOS ONE
In this report, we analyzed the performance of MAAs for identifying individuals with recent infection. This study was conducted using samples and data from the HIV Prevention Trials Network 071 trial (HPTN 071, PopART) [32]. This community-randomized trial was conducted in Zambia and South Africa and evaluated the impact of a combination prevention intervention that included universal testing and treatment on HIV incidence [32]. Early ART initiation could potentially confound performance of serologic incidence assays if a substantial portion of individuals with recent infection are virally suppressed. HPTN 071 (PopART) enrolled a randomly-selected Population Cohort (PC) of >48,000 participants who were followed for up to three years [32]. High rates of viral suppression were achieved in the PC after two years of study intervention [32]. We previously demonstrated that two MAAs provided accurate point estimates of HIV incidence in this cohort [33]. In this report, we evaluated whether four MAAs developed for population-level cross-sectional HIV incidence estimation could accurately identify individuals in HPTN 071 (PopART) who were infected <1 year. This cohort included many individuals who were virally suppressed early in infection. This allowed us to extend our analysis to include a comparison of the impact of natural vs. ART-induced viral suppression on the performance of serologic incidence assays.

Samples used for analysis
This study was conducted using samples and data from the HPTN 071 (PopART) trial (NCT 019000977) carried out in 21 study communities in Zambia and South Africa. Most HIV infections in the study countries are HIV subtype C. Samples were collected from participants in the PC at annual visits, referred to as PC0, PC12, PC24, and PC36. HIV status was determined for PC participants at each study visit at the HPTN Laboratory Center [34]. This report is based on analysis of samples collected at the PC24 visit. The analysis was limited to participants who had HIV status determined at the PC12 and PC24 visits (Fig 1); participants with acute HIV infection (RNA positive, antibody negative) at either of these visits were excluded from analysis. The analyses in this study included two groups of participants: (1) participants who were HIV negative at PC12 and HIV positive at PC24 (infected <1 year, seroconverters), and (2) participants who were HIV-positive at both the PC12 and PC24 visits (infected >1 year, non-seroconverters). Median duration between the PC12 and PC24 visits was 391 days (interquartile range [IQR]: 345, 436) for the seroconverter group and 382 days (IQR: 341, 421) for the non-seroconverter group. HIV status from the PC0 visit was available for a subset of the participants in the non-seroconverter group; these data were used to identify those infected for >2 years.

Laboratory methods
The testing described in this report was performed at the HPTN Laboratory Center (Baltimore, MD). Recency was assessed using four MAAs that included two or more assays (Table 1). These four MAAs were developed for population-level cross-sectional HIV incidence estimation. The HIV-1 Limiting Antigen (LAg)-Avidity EIA (Sedia Biosciences Corporation, Beaverton, OR; LAg-Avidity assay) was performed according to manufacturer's instructions. The JHU BioRad-Avidity assay was performed as previously described [35]; this assay is based on the Genetic Systems 1/2 + O ELISA (Bio-Rad Laboratories, Redmond, WA). The rapid LAg assay (Asanté HIV-1 Rapid Recency assay, Sedia Biosciences Corporation, Beaverton, OR) was performed according to manufacturer's instructions; results for all samples were read manually by a single laboratory technician. Samples where the long-term infected band was not detected  Table 1. Multi-assay algorithms evaluated in this report.

MAA Laboratory results used to identify recent infections Mean window period (days)
LAg MAA a LAg OD-n <1.5 plus viral load >1,000 copies/mL 130 The table shows the assays and cutoffs included in the four multi-assay algorithms (MAAs) evaluated in this report.
These MAAs were developed for cross-sectional incidence HIV estimation, not for individual-level recency assessments. The mean window period indicates the average time after infection that persons are classified as recent. a The LAg MAA is widely used for HIV incidence estimation [22].
b MAA-C has been used for HIV incidence estimation in prior studies [9]. were categorized as recent. Viral load testing was performed previously using the RealTime HIV-1 assay (Abbott Molecular, Des Plaines, IL) with a validated dilution procedure (limit of quantification: 400 copies/mL) [32]. Antiretroviral (ARV) drug testing was performed using an assay that detects 22 drugs in five drug classes [36]. Detection of one or more ARV drugs was considered to be indicative of ART.

Statistical methods
Sensitivity and specificity for classifying persons infected <1 year as recently infected were calculated using seroconversion status as the gold standard for classifying participants as recent vs. non-recent. False recent rates (FRRs) were calculated as the percentage of persons infected >2 years who were classified as recently infected [37]. Univariate analyses were performed using the t-test, chi-square test, or Fisher's exact test. Multivariate analysis was performed using linear regression analysis for continuous output variables and logistic regression analysis for binary output variables. The final multivariate regression models were determined using backward stepwise procedures. Statistical analysis was performed using R version 4.0.2 [38], the MASS package [39], and the epiDisplay package [40].

Informed consent
Written informed consent was obtained from participants prior to enrollment in HPTN 071 (PopART). The HPTN 071 (PopART) study was approved by the institutional review board and ethics committees at the London School of Hygiene and Tropical Medicine, the University of Zambia, and Stellenbosch University.

Study cohort
In HPTN 071 (PopART), 20,531 participants had HIV status determined at both the PC12 and PC24 visits (Fig 1). This included 4,452 participants who were HIV positive at both visits (non-serconverters) and 221 participants who were HIV negative at PC12 and HIV positive at PC24 (seroconverters). Fifty-six participants were excluded from the non-seroconverter group because they were missing data for HIV viral load, the LAg-Avidity assay, or the JHU BioRad-Avidity assay; one seroconverter was excluded because of missing viral load data. The final study cohort included 220 seroconverters and 4,396 non-seroconverters; 3,497 (79.5%) of the non-seroconverters were known to be infected >2 years. Viral load was �400 copies/mL at the PC24 visit for 63 (28.6%) of the seroconverters and 3,226 (73.4%) of the non-seroconverters. Datasets for the final seroconverter and non-seroconverter groups are provided (S1 and S2 Datasets, respectively).

Performance of MAAs for identifying persons with recent HIV infection
We evaluated the performance of four MAAs: (1) the MAA recommended by the US CDC that includes the LAg-Avidity assay and viral load (LAg MAA) [15,22,24]; (2) a MAA optimized for incidence estimation in populations with subtype C HIV that includes the LAg-Avidity assay, JHU BioRad-Avidity assay, and viral load (MAA-C) [9]; (3) a MAA that includes the rapid LAg assay and viral load (Rapid MAA) [23,41], and a MAA that includes the LAg-Avidity and JHU BioRad-Avidity assays without viral load (Alternate MAA [4]) ( Table 1). The mean window period for a MAA indicates the average time after infection that individuals are classified as recent. The recommended mean window periods for the LAg MAA, MAA-C, Rapid MAA, and Alternate MAA are 130 days [22], 248 days [9], 180 days [23], and 119 days [4], respectively ( Table 1). As a first step, each MAA was assessed for the ability to identify persons infected <1 year as recently infected ( Table 2)

Subgroups of seroconverters identified as recent vs. non-recent using different MAAs
We compared the groups of individuals that were classified as recent vs. non-recent using the four MAAs. Only 32 (15%) of the 220 seroconverters were classified as recent by all four MAAs (Fig 2). More detailed analysis was performed for the three MAAs that had lower FRRs (S1 and S2 Figs). We next compared testing outcomes for the LAg MAA and MAA-C (S1 Fig). The recent group identified by the LAg MAA was a subset of the recent group identified by the MAA-C. In contrast, different, overlapping groups of non-seroconverters were misclassified as The table shows the performance characteristics for four multi-assay algorithms (MAAs; see Table 1). The cohort included 220 persons who were infected for less than

Evaluation of assay performance for seroconverters with ART-induced viral suppression
As noted above, viral suppression from ART can lead to misclassification of persons with long-standing infection as recent when using MAAs that do not contain viral load as a biomarker; this most likely reflects down-regulation of anti-HIV antibodies. We next evaluated whether ART-induced viral suppression also impacts results obtained for seroconverters using serologic incidence assays. We compared the results obtained using the LAg-Avidity assay and JHU BioRad-Avidity assay in two participants groups: viremic seroconverters (VL>400 copies/mL; n = 157; five had ARV drugs detected, five did not have ARV drug data) and seroconverters with ART-induced viral suppression (VL�400 copies/mL + ARV drugs detected; n = 49) ( Table 3). The mean OD-n value obtained for these two groups was similar using the LAg-Avidity assay (viremic: 2.19 OD-n [standard deviation (SD): 1.47]; suppressed on ART: 1.96 OD-n [SD: 1.76]; p = 0.409). However, seroconverters who were suppressed on ART were more likely The associations noted above were also observed in multivariate models that included viral suppression status, sex, age, country, and study arm as input variables. Viral suppression on ART was independently associated with having a LAg OD-n value <1.5 (p = 0.0106; odds ratio [OR]: 2.36; 95% confidence intervals [CI]: 1.22, 4.55; viral suppression status was the only variable in this model after backwards selection). Viral suppression on ART was also independently associated with not having the long-term infected band on the rapid LAg assay (p = 0.0003; adjusted odds ratio [aOR]: 3.60; 95% CI: 1.77, 7.30; the final backward-selected model also included sex, country and study arm). In contrast, viral suppression on ART was independently associated with having a higher mean JHU BioRad-Avidity value (p = 0.033; difference in means: 10.5%, CI: 0.90%, 20.05%; viral suppression status was the only variable in this model after backwards selection); higher JHU BioRad-Avidity values are usually associated with longer duration of infection.

Comparison of assay performance for seroconverters with natural vs. ARTinduced viral suppression
We next compared results obtained with the LAg-Avidity and JHU BioRad-Avidity assays using samples from seroconverters with natural vs. ART-induced viral suppression (Table 4); samples were classified into these two groups based on results from the ARV drug assay. The table compares results obtained with the LAg-Avidity assay, Rapid Lag assay, and BioRad-Avidity assay for viremic seroconverters and seroconverters who were virally suppressed on ART. a Viral suppression was defined as having a viral load �400 copies/mL. b Laboratory detection of one or more ARV drugs was considered to be indicative of ART.
Abbreviations: AI: avidity index, ART: antiretroviral therapy, VL: viral load. https://doi.org/10.1371/journal.pone.0258644.t003 Participant groups included seroconverters with ART-induced viral suppression (n = 49, same group used in the analysis above) and virally-suppressed seroconverters with no ARV drugs detected (n = 13; one suppressed participant did not have ARV drug data and was excluded). There was no statistical difference in the mean values obtained for the LAg-Avidity-Assay

Discussion
This study compared the performance of four MAAs for individual-level recency assessments. These MAAs were developed for population-level HIV incidence estimation and were not optimized for individual recency assessments. All four MAAs classified <50% of the seroconverters as recent (range 24% to 45%), and only 15% of the seroconverters were classified as recent by all four MAAs. The length of the window period did not correlate with the number of seroconverters classified as recent. The MAA-C, which has the longest mean window period of the four MAAs, identified the highest number of seroconverters as recent. In contrast, the Alternate MAA, which has the shortest mean window period, classified more seroconverters as recently infected than two other MAAs. The FRR also varied for the four MAAs (from 0.2% to 1.3%). The two MAAs that classified the highest number of seroconverters as recent (the MAA-C and Alternate MAA) also had the highest FRRs. This illustrates the trade-off between identifying more recently-infected individuals and misclassifying more individuals with longterm infection as recently infected with some MAAs. In this study, the Rapid MAA classified the lowest number of seroconverters as recent, despite having a relatively high mean window period. Analysis of this MAA by the Consortium for the Evaluation and Performance of HIV Incidence Assays (CEPHIA) suggested that the mean window period for this MAA may be lower than the window period provided by the manufacturer [41], which could explain our findings.
We also found that the MAAs identified different subsets of seroconverters as recently infected. This was even observed for the LAg MAA and Rapid MAA; both of these MAAs include a LAg-based assay with the same target antigen and use the same viral load cutoff. Further research is needed to understand performance differences between the laboratory-based and point-of-care LAg assays.
ART-induced viral suppression impacted the performance of the LAg-Avidity and rapid LAg assays, but did not meaningfully impact the performance of the JHU BioRad-Avidity assay, consistent with prior studies [4,9]. The effect of viral suppression on the performance of the two LAg-based assays would not impact performance of MAAs that classify all virally-suppressed persons as non-recent. However, it may impact use of these assays in MAAs that do not include viral load. Interestingly, the performance of the LAg-Avidity assay differed in persons with natural vs. ART-induced viral suppression (individuals who were virally suppressed on ART had lower LAg-Avidity values than those who were virally suppressed in the absence of detectable ARV drugs). Further research is needed to understand the differences in serological responses between individuals with early viral suppression on ART and those who have low viral loads in the absence of ART.
One limitation of this study is that the study cohort only included individuals in a population survey from regions with predominantly subtype C HIV infection; results may differ in cohorts with other HIV risk factors and subtypes. Also, because study visits in HPTN 071 (PopART) were conducted annually, we were not able to assess performance of the MAAs for persons who were known to be infected for shorter periods. For sensitivity and specificity calculations, true positive results were defined as cases where the MAA identified persons infected <1 year as recently infected. Of note, all four MAAs have window periods less than a year (119 to 248 days) and would therefore not be expected to identify all persons infected <1 year as recently infected. In addition, as noted previously, the interval between the PC12 and PC24 visits in HPTN 071 (PopART) was not precisely one year. An additional limitation is that we only retested specimens with a LAg-Avidity result <2.0 OD-n, per manufacturer guidelines. These guidelines were developed to ensure assay reliability for a cutoff of 1.5 OD-n. This study included MAAs with a higher cutoff (2.8 OD-n), but guidelines for specimen retesting were not adjusted. Another limitation of this study is that the rapid LAg assay test strips were read manually by a single individual without validation by multiple readers for quality control; the manufacturers now recommend using an electronic reader to interpret test results. Further studies are needed to evaluate performance of the rapid LAg assay and Rapid MAA using manual vs. automated test interpretation.
This study highlights some of the limitations in using MAAs for individual-level assessments of the recency of HIV infection. These limitations should be considered before using MAAs for applications that include informing persons of their test results, since misclassification of the timing of infections could potentially lead to social harms. Further research is needed to develop MAAs that can accurately identify individuals with recent vs. nonrecent HIV infection. This study also demonstrates that the impact of viral load on incidence assays is different for those with natural vs. ART-induced viral suppression. Further research is needed to determine if this impacts performance of MAAs for HIV incidence estimation or other applications.